1. Field
Advancements in photogrammetry are needed to provide improvements in performance, efficiency, and utility of use.
2. Related Art
Unless expressly identified as being publicly or well known, mention herein of techniques and concepts, including for context, definitions, or comparison purposes, should not be construed as an admission that such techniques and concepts are previously publicly known or otherwise part of the prior art. All references cited herein (if any), including patents, patent applications, and publications, are hereby incorporated by reference in their entireties, whether specifically incorporated or not, for all purposes.
An example of a camera is an image capturing system that captures imagery using a lens that focuses light on at least one Petzval surface (e.g. a focal plane), and captures an image with at least one image sensor on the Petzval surface. A focal plane is an example of a planar Petzval surface. In general, Petzval surfaces are not required to be planar and may be curved due to the design of the lens. Examples of image sensors include film and electronic image sensors. Examples of electronic image sensors include Charge Coupled Device (CCD) sensors and Complementary Metal-Oxide Semiconductor (CMOS) sensors. An example of an emerging optical axis of a camera is the path along which light travels from the ground at the center of the lens field of view to arrive at the entrance to the camera. The light path inside the camera may be folded with reflecting surfaces, but eventually light arriving along the emerging optical axis will converge at the center of the Petzval surface(s).
Some maps assume a camera perspective looking straight down, called an orthographic (or nadir) perspective. In some embodiments and/or scenarios, this is also the perspective of the captured images used to make these maps (e.g. orthographic imagery). However, orthographic imagery eliminates all information about the relative heights of objects, and information about some surfaces (e.g. the vertical face of a building).
Other maps assume a camera perspective looking down at an angle below the horizon but not straight down, called an oblique perspective. An example of a down angle of a camera is the angle of the emerging optical axis of the camera above or below the horizon; down angles for nadir perspectives are thus 90 degrees; down angles for oblique perspectives are usually 20 to 70 degrees. In some embodiments and/or scenarios, the camera used to capture an oblique perspective is referred to as an oblique camera and the resulting images are referred to as oblique imagery. In some scenarios, oblique imagery is beneficial because it presents information that is useful to easily recognize objects and/or locations (e.g. height and vertical surfaces); information that is typically missing from orthographic imagery.
In some embodiments, the same point on the ground is captured with oblique images captured from multiple perspectives (e.g., 4 perspectives looking at a building, one from each cardinal direction: North, South, East, and West). This is sometimes described as ground-centric collection, and yields ground-centric oblique imagery. In various scenarios, ground-centric aerial oblique imagery is useful, e.g. for assessing the value of or damage to property, particularly over large geographic areas. It is usually a priority in a ground-centric collection program to collect an image of every point in some defined target area for each of the cardinal directions. The capture resolution is measured in distance units on the ground (e.g., 4 inch per pixel) and usually does not vary much between different points in the target area.
In some embodiments, multiple oblique images are captured from a single point, with multiple perspectives (e.g., 4 perspectives looking from a building in each cardinal direction), also known as sky-centric collection. In some scenarios, sky-centric imagery is commonly used to form a panoramic view from a single point. It is usually a priority in a sky-centric collection program to collect a continuous panorama from each viewpoint. Capture resolution is usually measured in angular units at the viewpoint (e.g., 20,000 pixels across a 360 degree panorama).
In various embodiments, a camera-group is a system of one or more cameras that approximately capture the same image (e.g. the optical axes are aligned within 5 degrees of a common reference axis). For example, an ordinary pair of human eyes acts as a 2 camera-group, focusing on a single image. Generally, a camera-group can have an arbitrary number of cameras.
In some embodiments, a camera-set is a system of one or more cameras and/or camera-groups that capture different images. One example of a 2 camera-set is a nadir camera and an oblique camera. Another example of a 4 camera-set is 4 oblique cameras, each pointing in a different cardinal direction. Generally, a camera-set can have an arbitrary number of cameras and/or camera-groups.
An example of the nominal heading of a vehicle is the overall direction of travel of the vehicle. In many scenarios, the instantaneous direction of travel deviates from the nominal heading. For example, an airplane may be flying along a flightpath heading due north, so that the nominal heading is north, while experiencing a wind blowing from west to east. To keep the plane on the flight path, the pilot will point the plane into the wind, so that the instantaneous heading is many degrees west of north. As another example, a car is driving down a straight road that runs from south to north and has several lanes. The nominal heading is north. However, to avoid hitting an obstacle, the car may changes lanes, instantaneously moving northwest, rather than strictly north. Despite this instantaneous adjustment, the nominal heading is still north. In contrast, when the car turns 90 degrees from north to travel west, the nominal heading is now west.
An example of a plan angle of an oblique camera on a vehicle is angle between the nominal heading of the vehicle and the emerging optical axis of the camera projected onto the ground plane; plan angles vary from 0-360 degrees. Some cameras are mounted on stabilization platforms so that the camera maintains its plan angle even as the instantaneous heading changes. Some cameras are mounted directly to the vehicle. Note that a vehicle may have a nominal heading, even when stopped, e.g. a helicopter with a flightpath due north could stop periodically, but would still have a nominal heading of due north.
Camera-sets used for sky-centric collection expend far more film (and later pixels) on ground points that the vehicle travels directly over, compared to ground points off to the side of the vehicle's path. When aerial photography and photogrammetry began to use airplanes, it became important to use less film to reduce costs. Some camera-sets removed the forward- and rear-facing oblique cameras of the earlier designs, and used a nadir camera and two oblique cameras pointing to the side (e.g. all emerging optical axes approximately perpendicular to the nominal heading of the airplane). While flying in a straight line and capturing overlapping images, these camera-sets capture the same amount of ground area with the same resolution as the more complex panoramic cameras and/or camera-sets, but with less film.
The extent of coverage in the direction of flight (sometimes described as in track) is primarily determined by the distance of flight. The extent of coverage orthogonal to the direction of flight (sometimes described as cross track) is primarily determined by the plane's altitude and the design of the camera. The extent of coverage in the cross track direction is sometimes called the swath. One benefit of a camera-set with both an oblique camera and a nadir camera is achieving greater swath without complex lens designs (e.g., a single large FOV fisheye).
In some sky-centric collection scenarios, the vehicle is maneuvered until the objects of interest are in view. For some ground-centric collection scenarios, the vehicle moves through a pattern which gives an opportunity to capture each point of interest on the ground from every required direction. In various embodiments, a Maltese Cross camera-set is moved in a path consisting of parallel lines (e.g. flight lines of an airplane) that run in a north-south or east-west direction. As the vehicle moves along the flight lines, the images captured by any particular camera can be superposed to form a long continuous strip of coverage. The length of this strip will be approximately the length of the flight line, and the width of this strip is known as the swath.
FIG. 1 conceptually illustrates an isometric view of selected prior art details of an airplane 102 with a Maltese Cross style oblique camera-set. The sensor fields of view of the forward 104, right 106, back 108, and left 110 oblique cameras are shown, projected onto the ground. The emerging optical axes of the cameras (respectively 112, 114, 116, and 118) have 45 degree down angles. Down Angle 122 is the angle formed between the Emerging Optical Axis 114 and its projection 120 to a plane parallel to the ground. For clarity, the other down angles are omitted from the illustration.
FIG. 2 conceptually illustrates a plan view of selected prior art details of the field of view of a single example camera of a Maltese Cross camera-set. The conical field of view projects from the camera aperture 208 to an ellipse 202 on the planar surface, with the longer major axis of the ellipse pointing away from the center of the camera. The image formed by the lens is a circle 210, which is shown at the left at a larger scale, and looking down the lens optical axis. The image sensor is an inscribed rectangle 212 that projects to a trapezoid 204 on the surface, because of the down angle of the camera. The image sensor is a rectangular array of pixels arranged in rows 220 and columns 216. The light rays 206 corresponding to the four corners of the image sensor are also shown. These light rays come from the ground up through the lens to the sensor. The pixels of the image sensor are projected onto the ground, forming projected rows 218 and projected columns 214. In this example, the rectangular image sensor is 24 mm by 36 mm, the focal length is 100 mm, and the camera altitude above the surface is 1000 meters. The resulting trapezoid is 455 meters wide at its base and 579 meters wide at its top.
FIG. 3 conceptually illustrates a plan view of selected prior art details of capturing oblique imagery via a Maltese Cross camera-set. In various embodiments, the nominal heading of the vehicle 301 is a cardinal direction (e.g. North, South, East, West). The camera-set includes four oblique cameras, with 0, 90, 180, and 270 degree plan angles. For conceptual clarity, the emerging optical axes are drawn in FIG. 3 with a 3 degree offset. Each camera has the same focal length and sensor size as the example camera in FIG. 2. However, the left and right cameras have the longer 36 mm dimension of the sensors aligned with the nominal heading. The projected FOV ellipses of the cameras 304, 308, 312, and 316 contain the projected sensor FOV trapezoids, respectively 302, 306, 310, and 314. Several captured images 320 of the projected FOV trapezoids are shown. The captured images from a single camera in a single flight line form a continuous strip, and there is significant forward overlap between images in the strip (e.g., at least 50% and typically 60% overlap between sequentially captured images).
The collection swath of a camera must fit within the projected FOV ellipses. In FIG. 3, the forward and back swaths are constrained by the minor axis of the front and back FOV ellipses; the side-facing swaths are constrained by the major axis of the side-facing FOV ellipses, which are significantly larger. In this example, the sensor FOVs of the left and right cameras are 487 meters wide, and the sensor FOVs of the front and back cameras are 458 meters wide (distance 355).
The swaths of the front- and rear-facing cameras are also significantly smaller than the separation between the swaths of the side-facing cameras. The front-facing camera swath is between edges 352 and 354, and as noted is 458 meters wide. The inner edges of the side facing swaths are denoted by edges 362 and 364, and the space between them 365 is 1571 meters.
FIG. 4 conceptually illustrates selected prior art details of an example flight plan for capturing oblique imagery covering Alexandria County, Virginia, using the Maltese Cross camera-set of FIG. 3. The flight plan 401 is arranged in 25 flight lines (e.g., 402) with nominal headings east or west, separated by 24 turns (e.g., 403) and captures oblique images that are oriented north, south, east and west. The total flight distance is 264 kilometers.
To capture the views offered by the front and rear facing cameras for every point of interest on the ground, the vehicle's flight lines must be closer together than the swath of the front and rear facing cameras. In the flight plan depicted in FIG. 4, the flight line pitch is 340 meters, so that there is 25% horizontal overlap between adjacent strips of imagery.